The world, as we experience it, is three-dimensional (3-D). However, although we experience a 3-D world, our senses do not directly receive 3-D data about the world. Instead, the optics of each eye project a two-dimensional (2-D) image onto the surface of the retina, and the visual system must infer data about the missing third dimension (i.e., depth) from those 2-D images and from various supplementary depth cues. These depth cues include the oculomotor cues of vergence and accommodation, and the stereoscopic cue of binocillar disparity.
Vergence: When a person's gaze is shifted to an object, the person's eyes move to fixate on the object, that is, to place the retinal image of that object on the center of each eye's retina (the fovea), where the resolution of the eye is the highest. Oculomotor cues involve the sensing of the position of muscles in and around the eyes. One oculomotor cue, vergence, refers to the phenomenon that lines of sight of the eyes are approximately parallel to one another when they are fixating on a very distant object, and the eyes rotate in toward each other (i.e., converge) as they fixate on closer objects. The brain receives sensory feedback regarding the relative eye positions, and this information serves as a depth cue.
Accommodation: Like most cameras, the human eye has a limited DOF. When viewing a real scene, not every object in the scene is in focus at any given time. Instead, the viewer accommodates (adjusts the focus of the eye) to bring objects at various distances into focus. For instance, if the viewer accommodates to an object that is one meter away, the retinal image of an object that is 20 meters away is blurry. The farther away an object is from the focus point of the viewer, the blurrier the retinal image of that object is.
The eye possesses a two part optical system. The cornea provides the majority of refraction (approximately 70 percent), but its refractive power is fixed. The crystalline lens sits behind the cornea, and its shape can be altered to increase or decrease its refractive power.
When a fixated object is close to the observer, the ciliary muscles of the eye contract in order to make the crystalline lens more spherical and increase its refractive power, so that the image of the object is brought into focus on the retina. When a fixated object is far from the observer, the ciliary muscles of the eye relax, thereby flattening the lens and decreasing its refractive power (measured in diopters) so that the image of the object is brought into focus on the retina. Dioptric blur provides negative feedback that the accommodation control system uses when trying to accommodate to an object at a given distance at a given point in time. If a person looks at an object at a novel depth, it will be blurred from the initially inaccurate state of accommodation. If the system begins to shift accommodation in one direction, and the object becomes more blurry, this blur feedback causes the system to reverse the direction of the accommodation shift. If, instead, the object becomes clearer, then accommodation continues to shift in the same direction. If the shift in accommodation overshoots the point of best focus, this manifests as increased blur, and the shift in accommodation reverses direction and slows. These shifts in accommodation continue until the blur feedback is minimized (the object comes into best focus). The process is dynamic, and the eye constantly monitors blur feedback and makes corrections in accommodation at rates up to 5 Hz. This process of natural viewing and focusing is known as closed-loop accommodation, because the blur feedback loop is intact (or “closed”). The brain receives feedback about the state of activity of the ciliary muscles, providing the viewer with information about the depth of the object being viewed.
Some viewing conditions artificially increase the DOF of the eye. For instance, if a scene is viewed through a small pinhole, then both distant and near objects are in focus at the same time. Under such conditions, the negative feedback of dioptric blur is removed or substantially decreased, and accommodation is said to be “open-loop” (because the feedback loop is interrupted). Under open-loop accommodation conditions, the viewer can accommodate from extremely near to far without a significant change in the retinal image of the scene. Some video displays can be made to have a very large DOF. As one example, the virtual retinal display (VRD) described in U.S. Pat. No. 5,467,104 can have a large DOF, producing an open-loop accommodative response in users. Other large DOF displays can be fabricated and methods presented in this document are applicable to all possible large DOF displays.
Vergence and Accommodation are Synkinetic: When one shifts one's gaze to an object at a given depth, the resultant vergence and accommodation responses are highly correlated. Not surprisingly, the accommodation and vergence mechanisms are synkintetic (an involuntary movement in accord with one mechanism is triggered when a movement in accord with the other mechanism occurs). This linkage can be observed in infants between three to six months old, suggesting a biological predisposition for the synkinesis. When the eye accommodates to a certain depth, the vergence system is automatically driven to converge to the same depth. Conversely, when the eye converges to a certain depth, the accommodation system is automatically driven to accommodate to the same depth. These cross couplings between accommodation and vergence are referred to as convergence driven accommodation and accommodation driven vergence.
Binocular Disparity and Stereopsis: Another depth cue is binocular disparity. Because a small distance separates the two eyes, they have slightly different viewpoints, and hence different retinal images. In stereopsis, the visual system compares the images from the left and right eye, and makes inferences about the depth of objects based on disparities between the retinal locations on which the images of the objects fall. This depth cue has been exploited in stereographic displays (including Head Mounted Displays (HMDs)), which present different images to each eye.
An object at the point of fixation falls on corresponding points of the retina (the center of the fovea, in this case). Other objects at approximately the same depth as the fixated object will also fall on corresponding points of the retina. The imaginary curved plane that describes the area of space that will fall on corresponding retinal points is referred to as the horopter. Objects behind the horopter will be shifted toward the left side of the retina in the right eye and toward the right side of the retina in the left eye (i.e., the images are shifted toward the nose). Objects in front of the horopter will be shifted toward the right side of the retina in the right eye and toward the left side of the retina in the left eye (i.e., the images are shifted toward the ears).
Interaction between Accommodation, Vergence, and Stereopsis: All of these depth cues interact. As mentioned previously, accommodation and vergence are synkinetic. Vergence and stereopsis interact. In order to stereoscopically fuse objects at different distances, the eyes must converge to fixate upon those objects. The relative distance between right and left object images is greater for objects in the foreground of a stereographic image than for objects in the background of the image. As viewers use stereographic displays and look at objects in the foreground and background of the displayed scene, they must dynamically shift vergence.
Accommodative Response to Current Non-Stereographic Video Displays: Research has indicated that viewers do not accurately focus their eyes on standard (non-stereographic) video displays (e.g., liquid crystal displays (LCDs) and cathode ray tubes (CRTs)). Their focus is, instead, biased in the direction of the resting point of accommodation (the degree of accommodation of the lens when a person is in a dark room or is otherwise deprived of an adequate visual stimulus). This resting point is not at the ciliary muscle relaxation point, which would produce a lens refractive power of 0 diopters, and varies between individuals. This inaccurate accommodation causes a video display to become somewhat blurred, and is thought to be a major contributor to the eye fatigue and headaches that often accompany prolonged video display use. It would thus be desirable to reduce these inaccuracies in accommodation and thereby reduce a cause of eye strain.
Stereographic Video Displays: A number of video display manufacturers have attempted to increase the immersion and amount of information in the display by creating stereographic displays (the term stereoscopic display is often used interchangeably with this term). One example of a stereographic display is the stereoscopic head mounted display (HMD). Typically, HMDs consist of a helmet or set of goggles, with a separate small LCD screen set in front of each eye. Lenses are mounted between each LCD and eye and are typically configured to place the image at optical infinity. The images displayed on each LCD are not identical, but instead represent slightly different camera viewpoints. The left LCD displays the left half of a stereo image pair and the right LCD displays the right half.
Another example of this display is the stereographic head tracked display (HTD). Two versions of HTD are common. With some HTDs, a user dons lightweight LCD shutter glasses with lenses that become opaque or transparent in synchrony with the frames displayed on a large table or wall mounted 2-D video display. When the shutter over the left eye is opened, the shutter over the right eye closes, and the left half of a stereoscopic image pair is flashed on the display. When the shutter over the left eye then closes, the shutter over the right eye opens, and the right half of the stereoscopic image pair is displayed. The opening of shutters alternates from side-to-side in quick succession, and the display synchronously shifts between showing left and right views, with every other frame displayed on the monitor reaching each eye.
In other implementations of an HTD, the user wears glasses in which the left lens is polarized along an axis orthogonal to that of the right lens (e.g., the left lens is polarized vertically, while the right is polarized horizontally). The user looks at a screen, upon which the left stereo image has been projected with light polarized along one axis, and upon which the right image has been projected with light polarized along the other axis. Each lens admits only the light reflected from the screen that is of the matching polarization, so each eye is presented with a different image. Other implementations of stereographic displays include autostereoscopic displays, which enable users to view stereo images without wearing any special glasses.
With all of these stereoscopic displays, the left eye receives only the left stereo image, while the right receives only the right stereo image, giving the user the illusion that he/she is looking into a virtual scene with two eyes at normal eye separation.
Current Stereographic Displays Violate Accommodation-Vergence Synkinesis: Current stereographic displays, especially HMDs, tend to cause profound eye fatigue, often accompanied by headache. Research suggests that this eye fatigue is, in part, the result of an incompatibility between the construction of the display and the biology of accommodation. Specifically, the displays elicit a mismatch between the depth information provided by accommodation and that provided by vergence. The displays have a fixed plane of focus (usually at optical infinity, or 0 diopters) making it necessary for the viewer's eyes to maintain a static level of accommodation while using the display. If the viewer shifts his/her level of accommodation, the display becomes out of focus. Nonetheless, stereo displays also require that the viewer dynamically change the vergence of his/her eyes to binocularly fuse objects at different apparent distances. Accommodation and vergence are yoked—that is, they are synkinetically linked, such that when a person's eyes converge, the eyes also tends to accommodate near, and when a person's eyes deconverge, they tends to accommodate far. These displays violate this linkage and require a viewer to simultaneously converge his/her eyes while maintaining a fixed level of accommodation. Accordingly, it would be desirable to provide a display that allows accommodation and vergence to shift synchronously, just as they do in natural viewing, and thus removes the primary source of stereo display eye strain and loss of visual acuity.